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Abstract 


We describe the formal specification and verification of an algorithm 
for Interactive Consistency [12] based on the Oral Messages algorithm for 
Byzantine Agreement [9]. We compare onr treatment with that of Bevier 
and Young [2,3], who presented a formal specification and verification for 
a very similar algorithm. Unlike Bevier and Young, who observed that 
“the invariant maintained in the recursive subcases of the algorithm is 
significantly more complicated than is suggested by the published proof” 
and who found its formal verification “a fairly difficult exercise in me- 
chanical theorem proving,” our treatment is very close to the previously 
published analysis of the algorithm, and our formal specification and 
verification are straightforward. 

This example illustrates how delicate choices in the formulation of 
a problem can have significant impact on the readability of its formal 
specification and on the tractability of its formal verification. 




II 



Contents 


1 Introduction 1 

2 Informal Overview 4 

2.1 Interactive Consistency , 4 

2.1.1 Oral Messages 4 

2.1.2 The Original Algorithm 5 

2.2 Byzantine Generals 6 

2.2.1 The Correctness Argument 7 

3 Bevier and Young’s Verification 9 

4 Specification and Verification in EHDM 13 

5 Discussion and Conclusion 21 

5.1 Discussion 21 

5.2 Conclusion 23 

Bibliography 25 

A The “Real” Specifications 28 

B The Byzantine Generals Formulation of the Algorithm 31 

C The Full Specification and Verification 33 

C.l The Specification 33 

C.1.1 Module “Consensus” 33 

C.2 Proof- Chain Analysis 42 


iii 


P RECEDING PAGE BLANK NOT FILMED 



t 


List of Figures 


3.1 Bevier and Young’s Specification of the Oral Messages Algorithm . . 10 

3.2 Bevier and Young’s “Invariant” for BG2 12 

4.1 Our specification of the Oral Messages Algorithm 17 

A.l Bevier and Young’s Specification — The Real Version ......... 29 

A. 2 Our Specification — The Raw Text Version 30 

B. l Our Formulation of the Byzantine Generals Version of the Algorithm 32 


IV 


Chapter 1 

Introduction 


Fault tolerant systems, such as those used in digital flight control, require a way 
to ensure that the replicated processors all work on the same input values. For 
example, each processor may sample different sensors (or the same sensor at different 
times) and thereby obtain different estimates of some external value; these different 
estimates need to be combined into a single consensus value that is the same for 
all processors. By starting with the same inputs, all correctly working processors 
should then compute the same outputs and faults can be masked using exact-match 
majority voting. 

The problem of deciding on a single consensus value can be broken into two 
stages. In the first stage, the processors exchange their private data values among 
themselves. At the end of this stage, each processor has a vector giving the data 
values of all the other processors; if there are no faults, these vectors will be identi- 
cal on all processors. The second stage may then comprise any data conditioning, 
selection, or averaging algorithms whatever: provided all processors run the same al- 
gorithms, and start with the same vectors, they will end up with the same consensus 
values. 

We are interested in the first stage of this process, and with ensuring that it 
performs reliably in the presence of arbitrary faults. (The worst kinds of fault 
are “asymmetrical” ones where a faulty processor communicates different values 
to different processors, potentially causing nonfaulty processors to disagree among 
themselves.) This problem of reaching agreement in the presence of faults was first 
posed, named, and solved by Pease, Shostak, and Lamport in 1980 [12]. They named 
the problem that of achieving “Interactive Consistency.” In 1982, the same authors 
developed their analysis further, and reformulated it as the “Byzantine Generals 
Problem” [9]; they named a revised version of the algorithm from their earlier paper 
the “Oral Messages” algorithm. The principal difference between the Interactive 
Consistency and Byzantine Generals problems is that the former is concerned with 
the reliable exchange of values among all the participants, whereas the latter is con- 
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cerned with the reliable communication of a value from a distinguished participant 
(called the “General”) to all the others (who are called “lieutenants”). In practi- 
cal applications, it is the Interactive Consistency formulation that is appropriate, 
but the colorful metaphor of the Byzantine Generals has proved so memorable that 
this formulation is better known; indeed, the whole field of algorithm design for 
agreement in the presence of faults has become known as that of “Byzantine Agree- 
ment,” and the asymmetrical kind of fault mentioned earlier has become known as 
a “Byzantine fault.” 

A problem related to Interactive Consistency is Byzantine fault-tolerant clock 
synchronization [8]. In 1988, we formally verified the “Interactive Convergence” 
algorithm for this problem [8, Algorithm CNV] and found that the published anal- 
ysis of this algorithm was incorrect in a number of details [15,16]. Our colleague 
Shankar has formally verified the generalized clock synchronization paradigm of 
Schneider [18] and similarly found a number of small errors [19,20]. In both cases, 
the formal verification led to improved and simplified presentations of the infor- 
mal justifications for the correctness of the algorithm concerned. We have often 
wondered whether formal verification of the Oral Messages algorithm for Byzan- 
tine Agreement would yield similar benefits, and have been curious to know how 
difficult the formal verification of this algorithm would be, compared to the clock 
synchronization algorithms. 

In 1990, a formal verification of the Oral Messages Algorithm was published by 
Bevier and Young [3] as part of the documentation of a more substantial exercise 
in which they also verified the design of a circuit to perform the algorithm, and the 
theorem that the fault-tolerance of the Oral Messages Algorithm is optimal among 
its class of algorithms. 

Bevier and Young described the algorithm as “quite difficult” and have indi- 
cated elsewhere that development of its formal verification (using the Boyer-Moore 
prover [4]) took them about a month. We found this surprising, since the published 
journal proof for the correctness of the Oral Messages algorithm [9, page 390] is 
short (less than a page) and straightforward. The time taken may be explained by 
Bevier and Young’s observation [3, page 1] that their machine-checked proof 

“. . .elucidates several issues which are treated rather lightly in the pub- 
lished version of the proof. In particular, the invariant maintained in 
the recursive subcases of the algorithm is significantly more complicated 
than is suggested by the published proof.” 

After careful study of Bevier and Young’s presentation, however, we were unable to 
persuade ourselves that their claim of suppressed complexity in the published journal 
proof is justified. On the contrary, we continued to find the journal description and 
proof more compelling than their formal presentation. In order to resolve our doubts, 
we decided to undertake a separate formal verification using our Ehdm system [17]. 
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There are relatively few examples of interesting or difficult verifications under- 
taken by more than one group, or using more than one system for formal specifica- 
tion and verification. Bill Young’s comparison of Z and Gypsy [24] and the 12-way 
comparison reported by Jeannette Wing [23] are concerned solely with specification. 
Rather more interesting are David Basin and Matt Kaufmann’s comparison of two 
verifications of the finite Ramsey theorem [1], and Bill Young’s duplication [25] of 
our verification [15,16] of a clock synchronization algorithm [8]. 

One reason for the paucity of comparisons using substantial or difficult examples 
is that only a handful of verification systems are capable of undertaking such exam- 
ples, and the developers and users of those systems are fully engaged in their own 
lines of enquiry. When they can be performed, however, such comparisons are very 
useful, since they provide the only reasonable way to compare claims for “readabil- 
ity” or “expressiveness” in specification languages, and “power” or “effectiveness” 
in verification environments. 

Comparative studies can be undertaken at several different levels: two dif- 
ferent systems can be used to proof-check the same verification (cf. the clock- 
synchronization example mentioned above); two different verifications can be per- 
formed for the same specification; two different formalizations can be developed for 
the same specification (cf. the study reported by Jeannette Wing); or two completely 
separate formal developments can be performed for a single problem. Different 
lessons are likely to be learned from these different levels of comparison: when one 
tool or notation is simply substituted for another, we may learn something about the 
ability of the second to duplicate the results of the first on its “home ground,” but 
we will not learn how the problem might have been approached differently had the 
second tool or notation been used from the start; and when two independent devel- 
opments are undertaken, we may learn more about the problem-solving approaches 
of the individuals concerned than about the tools employed. 

The experiment described here is of the latter kind, and it may be that the main 
conclusion to be drawn concerns the considerable impact that apparently small 
changes in the formulation of a problem can have on the tractability of its formal 
verification. On the other hand, this example also invites speculation on the benefi- 
cial influence that an expressive specification language and a direct approach to proof 
may have in the development of felicitous formulations of interesting algorithms. 



Chapter 2 

Informal Overview 


In this section we briefly review the Interactive Consistency (IC) and Byzantine Gen- 
erals (BG) problems, and the “Original” (OA) and Oral Messages (OM) algorithms 
for solving them. We follow the presentations of Pease, Shostak, and Lamport [9,12] 
very closely. 


2.1 Interactive Consistency 

Consider a set of n isolated processors, of which some may be faulty. It is not known 
which processors are faulty, nor how many, nor what behavior may be exhibited by 
faulty processors. Suppose also that each processor p has some private value v p 
(such as its reading of some sensor). The problem is to devise an algorithm that will 
allow each processor p to compute a vector V v of values, in which, for each processor 
r, V p (r ) is p’s estimate of r's private value, satisfying the following conditions: 

IC1: If processors p and q are nonfaulty, then they agree on the value ascribed to 
any other processor r; that is: V p (r) — V q (r). 

IC2: If processors p and r are nonfaulty, then the value ascribed to r by p is indeed 
r’s private value; that is, V p (r) = v T . 

2.1.1 Oral Messages 

There are many variations on the IC and BG problems that differ in the assumptions 
made about interprocessor communications. For example, whether the processors 
are fully connected, whether messages can be lost, and whether a faulty processor 
can forge a message purporting to have come from another. The Oral Messages 
assumptions are: 

Al: Every message that is sent between nonfaulty processors is correctly delivered. 
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A2: The receiver of a message knows who sent it. 

A3: The absence of a message can be detected. 

An algorithm based on Oral Messages solves the IC problem under these as- 
sumptions. The principal difficulty that must be overcome by such an algorithm is 
that a faulty processor may send different values to different nonfaulty processors, 
thereby complicating satisfaction of condition ICl. To overcome this, an algorithm 
will use several “rounds” of message exchange during which processor p tells pro- 
cessor q what value it received from processor r and so on. Of course, if processor 
p is faulty, it may “be” about the value it received from processor r. By making 
sufficiently many rounds, an algorithm can defeat this threat. 

2.1.2 The Original Algorithm 

The original algorithm [12, page 230], which we will abbreviate as OA, is parame- 
terized by n, the number of processors, and m (where n > 3m + 1), the maximum 
number of faulty processors. The following description of OA is taken verbatim 
from [12, page 230] (except that we have changed V to v). 

“Let P be the set of processors and v a set of values. For k > 1, we 
define a k-level scenario as a mapping from the set of nonempty strings 
(possibly having repetitions) over P of length < k + 1, to v. For a given 
fc-level scenario, a and string w = p\p 2 . . .p r , 2 < r < k + 1, a(w) is 
interpreted as the value p 2 tebs pi that p 3 told p 2 that p 4 told p 3 . . .that p r 
told p r - 1 is p r ’ s private value. For a single- element string p, a (p) simply 
designates p’s private value v p . A A-level scenario thus summarizes the 
outcome of a fc-round exchange of information. (Note that if a faulty 
processor bes about who gave it information, this is equivalent to lying 
about a value it was given.) Note also that for a given subset of nonfaulty 
processors, only certain mappings are possible scenarios; in particular, 
since nonfaulty processors are always truthful in relaying information, a 
scenario must satisfy 

o(pqw) = cr(qw) 

for each nonfaulty processor q , arbitrary processor p, and string w. 

“The messages a processor p receives in a scenario a are given by 
the restriction o v of a to strings beginning with p. The procedure we 
present now for arbitrary m > 0, n > 3 m + 1, is described in terms of p’s 
computation for a given <r p , of the element of the interactive-consistency 
vector corresponding to each processor q (i.e., V p (q)). The computation 
is as follows: 
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1. If for some subset Q of P of size > (n + m)/2 and some value u, 
Op(pwq) = v for each string w over Q of length < m, p records u. 

2. Otherwise the algorithm for m— 1, n — 1 is recursively applied with 
P replaced by P - {^}, and o v by the mapping o v defined by 

cr p (pw) = o p (pwq) 

for each string w of length < m over P-{q}. If at least [(n + m)/2j 
of the 72 — 1 elements in the vector obtained in the recursive call 
agree, p records the common value; otherwise p records NIL. 

Note that o v corresponds to the m-level subscenario of o in which q 
is excluded and in which each processor’s private value is the value it 
obtains directly from q in a.” 

We expect that many readers will share our opinion that this description of 
OA is a challenge to comprehension. The argument for its correctness [12, page 
231] is similarly hard to follow. The original authors also may have considered the 
presentation somewhat difficult, for a couple of years after the original publication 
they reformulated the problem, the algorithm, and the argument for its correctness. 
The revised presentation was couched in the metaphor of “Byzantine Generals” and 
is described in the next section. 

2.2 Byzantine Generals 

As mentioned earlier, BG differs from IC in that there is a distinguished processor 
called the General whose value is to be communicated to all other processors (called 
lieutenants ), 1 Again, there are n processors in total, of which some (possibly includ- 
ing the General) may be faulty. The General has some “order” v and the problem 
is to devise an algorithm that will allow each Lieutenant p to compute an estimate 
v v of the General’s order satisfying the following conditions: 

BGl: If Lieutenants p and q are nonfaulty, then they agree on the value ascribed 
to the General; that is v v = v q . 

BG2: If the General is nonfaulty, then every nonfaulty lieutenant has the correct 
order; that is v v = v. : .. 

We have renamed these conditions BGl and BG2 to distinguish them from the 
corresponding conditions of the IC case. 

Lamport, Shostak and Pease [9] often speak of the “Commanding General,” and refer to the 
others as the “lieutenant generals.” 


2.2. Byzantine Generals 
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The Oral Messages (OM) algorithm solves the BG problem under the same 
assumptions as OA; it can be regarded as a substantial reformulation of OA, rather 
than an independent algorithm. In order to distinguish the BG version of the 
algorithm from the IC version to be introduced later, we denote them OMBG and 
OMIC, respectively. The algorithm is characterized by the number of rounds to be 
made: OMBG(m) is the instance of the algorithm that makes m + 1 rounds. The 
following description is taken verbatim from [9, page 388]. Note that under the 
Byzantine Generals metaphor, faulty processors are called “traitors,” and nonfaulty 
ones are “loyal.” First we describe the simplest case, OMBG(O): 

OMBG(O) 

1. The General sends his value to every lieutenant. 

2. Each lieutenant uses the value he receives from the General, or uses 
the value retreat if he receives no value. 

Now we can describe the general case. 

OMBG(m), m > 0 

1. The General sends his value to every lieutenant. 

2. For each i, let v l be the value Lieutenant i receives from the General, 
or else be retreat if he receives no value. Lieutenant i acts as the 
General in Algorithm OMBG(m — 1) to communicate the value u, 
to each of the n — 2 other lieutenants. 

3. For each i, and each j ^ i, let vj be the value Lieutenant i received 
from Lieutenant j in step (2) (using Algorithm OMBG(m - 1)), or 
else retreat if he received no such value. Lieutenant i uses the value 
majority (v u ..., v n . x ). 

2.2.1 The Correctness Argument 

The argument for the correctness of OMBG is taken verbatim from [9, page 390] 

Lemma 1 For any m and k, Algorithm OMBG(m) satisfies BG2 if 
there are more than 2 k + m participants and at most k traitors. 

Proof: The proof is by induction on m. BG2 only specifies what 
must happen if the General is loyal. Using Al, it is easy to see that the 
trivial algorithm OMBG(O) works if the General is loyal, so the lemma 
is true for m = 0. We now assume it is true for m - 1, m > 0, and prove 
it for m. 

In step (1), the loyal General sends a value v to all n— 1 lieutenants. 

In step (2), each loyal lieutenant applies OMBG(m - 1) with n — 1 gen- 
erals. Since by hypothesis n > 2k + m, we have n - 1 > 2k + (m - 1), 
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so we can apply the induction hypothesis to conclude that every loyal 
lieutenant gets vj = v for each loyal Lieutenant j. Since there are at 
most k traitors, and n — 1 > 2k + (m — 1) > 2k , a majority of the n — 1 
lieutenants are loyal. Hence, each loyal lieutenant has Vi — v for a ma- 
jority of the n — 1 values z, so he obtains majority (v i, . . . , u n _i) = v in 
step (3), proving BG2. □ 


Theorem 1 For any m, Algorithm OMBG(m) satisfies conditions BGl 
and BG2 if there are more than 3 m participants and at most m traitors . 

Proof: The proof is by induction on m. If there are no traitors, then it is 
easy to see that OMBG(O) satisfies BGl and BG2. We therefore assume 
that the theorem is true for OMBG(m — 1) and prove it for OMBG(m), 
m > 0. 

We first consider the case in which the General is loyal. By taking 
k equal to m in Lemma 1, we see that OMBG(m) satisfies BG2. BGl 
follows from BG2 if the General is loyal, so we only need verify BGl in 
the case the General is a traitor. 

There are at most m traitors, and the General is one of them, so at 
most m — 1 of the lieutenants are traitors. Since there are more than 3m 
generals, there are more than 3m -1 lieutenants, and 3m — 1 > 3(m — 1). 
We may therefore apply the induction hypothesis to conclude that 
OMBG(m- 1) satisfies conditions BGl and BG2. Hence, for each j, any 
two loyal lieutenants get the same value for vj in step (3). (This follows 
from BG2 if one of the two lieutenants is Lieutenant j, and from BGl 
otherwise). Hence, any two loyal lieutenants get the same vector of values 
Vi 7 . . . , v n -i 7 and therefore obtain the same value majority (v i, . . . , v n -i) 
in step (3), proving BGl. □ 


Chapter 3 

Bevier and Young’s Verification 


Bevier and Young [3] performed a formal specification and verification of the OMBG 
Algorithm using the Boyer-Moore theorem prover [4], Insofar as the restrictions of 
the Boyer-Moore logic allow 1 , Bevier and Young’s specification and verification fol- 
lows the published version of Lamport, Shostak and Pease [9] very closely. Since 
the problem of practical interest is IC rather than BG, they augment their descrip- 
tion [3, Section 3.4] with the specification and verification with an additional step 
that applies OMBG iteratively (with each process in turn taking the role of the 
General), thereby extending it to a solution for IC. 

Bevier and Young specify OMBG in terms of two mutually recursive functions, 
vom* and voml*; the former is the main OMBG function, while the latter specifies 
the iterative application over all lieutenants required in step (2) of the former. In 
addition, the function vomO specifies the base case OMBG(O). These functions are 
reproduced in Figure 3.1 (taken from [3, Figure 5, page 7]). 2 

Bevier and Young explain these functions as follows [3, pages 6,7]. Note that 
the function (send v i j) denotes the value received when process i sends value v 
to j . 


“vom* is the top-level function which takes as arguments the number 
m of rounds, the General’s name g and value v, a list 1 of lieutenant 
names, and the vector vec in which the message traffic is recorded. 

It returns a vector in which each lieutenant’s position is filled by that 
lieutenant’s view of the General’s value. Arriving at this view requires 
77i — 1 rounds of communication (the call to the voml* function) com- 
bined (pair’d) with the initial round in which the General distributes 

J The Boyer-Moore logic is an untyped, unquantified first-order logic resembling pure lisp. 

2 In order to satisfy the definitional principle of the Boyer-Moore system, the mutually recursive 
pair vom* and voml* are encoded in the actual specification as a single function with a “flag” 
argument to distinguish the two cases. This version is reproduced in Appendix A, Figure A.l. 
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Definition 
(vomO g v 1 vec) 

(if (listp 1) 

(put (car 1) 

(send v g (car 1)) 

(vomO g v (cdr 1) vec)) 

vec) 

Definition 
(vom* m g v 1 vec) 

(if (zerop m) 

(vomO g v 1 vec) 

(votelist 

(pair (vomO g v 1 vec) 

(voml* (subl m) 1 (vomO g v 1 vec) 1 vec) 

1 ») 


Definition 

(voml* m g-list vomO 1 vec) 

(if (listp g-list) 

(pair (vom* m (car g-list) (get (car g-list) vomO) 
(delete (car g-list) 1) vec) 

(voml* m (cdr g-list) vomO 1 vec) 

(delete (car g-list) 1)) 

(init nil (length vec))) 


Figure 3.1: Bevier and Young’s Specification of the Oral Messages Algorithm 
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his value directly (the call to vomO), and voting on each element in the 
resulting map (the call to votelist). 

“The function voml* takes as arguments the number m of exchanges, 
a list g-list of names of processes which will serve in turn as the general 
in this round, a vector vomO in which each process’s slot is filled with its 
value sent to it by the General, a list 1 of the other lieutenants, and a 
vector vec in which the message traffic is recorded. It returns a vector 
in which each lieutenant’s name is bound to the list of messages that 
lieutenant has received in this round of message exchanges.” 

Bevier and Young state that the verification that their specifications of OMB G 
satisfy BGl and BG2 is “a fairly difficult exercise in mechanical theorem prov- 
ing” [3, page 1] but that they “gained considerable insight into the algorithm” from 
their formalization [3, page 13]. They illustrate the latter point by referring to the 
published proof of OMBG (reproduced in Section 2.2.1 above) and observing: 

“Though seemingly straightforward, there is a considerable degree of 
suppressed detail in this proof. In particular, the induction hypothesis 
refers to what happens after each round of message exchange without 
worrying about the intermediate states which occur during each round. 

In terms of our mutually recursive version of the algorithm, the proof 
above describes the induction by referring to what happens after each 
call to vom* and simply assumes what happens in the calls to voml*. 

“What happens in these calls, and what is crucial from the point of 
view of a fully formal proof, is that there is a rather involved invariant 
maintained by the algorithm. A key part of this invariant can be stated 
roughly as follows: after each round of message exchange all of the non- 
faulty processors agree on a value for the General, that value being the 
General’s actual value. This notion we call non-faulty agreement . 

“Formulating and proving an appropriate version of the invariant for 
BG2 was the primary effort in the proof.” 

The invariant referred to above is reproduced in Figure 3.2. Bevier and Young “do 
not bother to describe some of the subsidiary concepts such as non-f aulty-value 
which are involved in the statement of the invariant” [3, page 14] and do not ex- 
hibit the corresponding invariant for BGl, but note that it “is substantially more 
involved.” 
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Theorem. VOM-IC2-INVARIANT 
(implies 

(and (setp 1) 

(bounded-number-listp 1 (length vec)) 

(member i 1) 

(not (faulty i))) 

(if fig 

(implies 

(and (not (member g 1)) 

(not (faulty g)) 

(leq (plus (times 2 (fault-count 1)) m) 
(length 1))) 

(equal (get i (vom fig m g v 1 vec)) 
v)) 

(implies 

(and (subbagp g 1) 

(equal (length v) (length vec)) 

(lessp (plus (times 2 (fault-count 1)) m) 
(length 1)) 

(non-faulty- agreement (non-f aulty-value g v) 

g v)) 

(not (lessp (occurrences 

(non-f aulty-value g v) 

(get i (vom fig m g v 1 vec))) 

(if (member i g) 

(subl (good-count g)) 

(good-count g))))))) 



Figure 3.2: Bevier and Young’s “Invariant” for BG2 



Chapter 4 

Specification and Verification 
in EHDM 


One source of complexity in both the specification and verification of Bevier and 
Young’s formulation of OMBG is the need for a pair of mutually recursive functions. 
An additional burden is the need to perform a second specification and verification 
in order to connect BG to IC. Both of these difficulties can be avoided by developing 
a version of OM that solves IC directly. One way to see that this approach is likely 
to be beneficial is to observe that the iterated recursion inside OMBG is solving an 
instance of IC: after the General has transmitted his value to all the lieutenants, each 
of those lieutenants has a private value (the value he received from the General), and 
the subgoal is for the n — 1 lieutenants to perform IC on those private values. Each 
lieutenant will then have an IC vector that gives the value sent by the General to 
each lieutenant; all nonfaulty lieutenants will have the same IC vector, and selecting 
the majority value from those vectors will cause each of them to assign the same 
value to the General. 

It follows that a generalization of OMBG from BG to IC should be simpler than 
OMBG, since the recursive subproblems will be the same as the parent. We will 
call this generalization the OMIC algorithm. We present the algorithm and the 
argument for its correctness in the next few pages. All the specifications that follow 
in this section are taken directly from our formal verification, and are in the language 
of Ehdm [17]; the proof sketches are also taken from our formal verification. The 
full specification and verification is presented in Appendix C. 

We will specify OMIC as a function of three arguments: m the number of rounds, 
v a vector giving the private values of each processor, and caucus the set of processors 
participating in the algorithm. Processors are represented by natural numbers in 
the range 0 . . . n — 1, and vectors are functions from processors to values (of some 
uninterpreted type T ). OMIC will return a “vector” of vectors: that is a function 
from processors to vectors. Thus OMIC(m, v y caucus)(p) will be the IC vector of 
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processor p following the OMIC algorithm, and OMIC(m, v , caucus)(p)(q ) will be 
p's opinion of q’s private value (i.e., of u(g)). Notice that we are using higher-order 
functions (i.e., functions whose values are functions) here. We have found higher- 
order constructions very convenient in several specifications that we have undertaken 
(see for example [13]). 1 

In preparation for formally specifying OMIC, we first state its behavior for the 
case m = 0. In this and the formulas that follow, free variables are treated as uni- 
versally quantified at the outermost level, and we do not generally identify the types 
of the variables appearing in these formulas (see the full specification in Appendix C 
for these subsidiary declarations). 


OMIC(0, v, caucus)(p)(?) 

= if p E caucus A q € caucus then send(v(g), q,p ) else undef end if 


Here, undef is some arbitrary value and send(v(q),q,p) is, as in Bevier and Young’s 
formulation a function that represents the value received by p when q sends it the 
value v(q). Our requirement on OMIC in the case m = 0 simply states that if p 
and q are both participants to the algorithm (i.e., both in the set caucus ), then p’s 
opinion of q's private value v(q) following the algorithm should be send(v(q),q,p). 

The property assumed of send is captured in the following axiom 


send.ax: Axiom ok(p) Aok(<?) D send (t,q,p) = t 


where ofc(p) is the predicate that asserts that processor p is nonfaulty. (We regard 
a processor that is faulty at any point in the algorithm as being faulty throughout.) 
Essentially, this axiom captures Assumption A 1 of oral messages. Notice that if 
either p or q are faulty, we know nothing whatever about the value send(t,q,p). 
Well, not exactly nothing: we do know that its value is functionally determined by 
t, p, and 9 . Thus, if q were to send < to p in a later round, the value received would 
be the same as in this round, whatever the fault-status of the processors concerned. 
This may not be realistic if p or q are faulty, so we will reformulate send to take the 
round number as an argument: send(r,t,q,p) then represents the value received by 
p when q sends it the value t in round r. The round number does not affect the 
transmission when nonfaulty processors are involved: 


send.ax: Axiom ok(p) Aok(g) D send (r,t,q,p) = t 


1 Higher-order functions are also used in Ehdm to specify set operations, which appear frequently 
in this specification. Sets are specified as their characteristic predicates in Ehdm and the operation 
that, for example, removes a processor from a set of processors has the specification 

caucus - {?}: function [set, processors — set] = ( A caucus, q : caucus with [(5) := false]) 
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The only effect and purpose of this modified treatment of the send function is to 
make it more clear that no assumptions at all are made about values communicated 
when either the sender or receiver is faulty. 2 

The specification of the behavior of OMIC in the case m — 0 needs to be adjusted 
accommodate the changed functionality of send: 

OMIC(0, v , caucus)(p)(<?) 

= if p E caucus A q £ caucus then send(0, <j,p) else undef end if 

For the case m = r, r > 0, we require that p’s opinion of g’s private value should 
be send(r, v(q),q, q) if p = q , 3 otherwise it should be the majority value in p’s IC 
vector, after performing OMIC with m = r — 1 on the current set of processors with 
q excluded, and the values received from q as the private values. Thus we require 

r > 0 D OMIC(r, v, caucus)(p)(<?) 

= if p G caucus A q £ caucus 
then if p = q 

then send(r, v(q) } q , q ) 

else maj(caucus - {q}, OMIC(r - 1, distr(r, v(q ), q), caucus - {q})(p)) 
end if 
else undef 
end if 

Here, distr(r, v(q) y q) is simply a function that uses send in round r to distribute 
the value v(q) from q to every other process: 4 

distr: function[rounds, T, processors — ► vector] = (Ar,i,p: (\z : send(r, t , p, z))) 

2 Even the modified formulation of send is not free of the suggestion that the value delivered by 
a faulty processor is functionally determined. The only way to completely remove this taint is to 
use a relational specification for send , with the interpretation that send(ii , g,p, <2) is true if p could 
possibly receive <2 when q sends it ti, and the axiom: 

send.ax: Axiom ok(p) A ok(g) 3 send(Z, q, p, t) 

The problem with this reformulation is that it has ramifications throughout the specification, re- 
quiring the definition of the Oral Messages algorithm itself, as well as several subsidiary functions, 
to become relations also. Our colleague Shankar has recently developed such a specification and 
verification of OMIC, but here we prefer to stay with the simpler, if slightly flawed, functional 
definition of send. 

3 We could specify v(q) in this case; we have chosen the weaker assumption that a faulty processor 
may not even know its own value. 

It might be less “wasteful” to add the set of recipient processors (i.e., caucus — {5}) as an addi- 
tional argument to distr , rather than have the value sent to every process. This sort of “economy” 
would be important in an implementation of the algorithm, but would clutter the specification and 
proof. 
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The function maj takes a set caucus of processors, and a vector v , and computes the 
majority value (if any) in that vector over that set. Actually, requiring this function 
to be implemented by a majority vote overspecifies the problem. All that is really 
required is specified in the following axiom, which states that if the good processors 
form a majority in caucus, and if all the good processors have the same value in the 
vector, then that is the value of the maj function. Notice that taking the median 
of the values of the members of caucus (assuming they come from an ordered set) 
would also satisfy this specification (as was correctly noted by Lamport, Shostak 
and Pease [9, page 388]). 

majax: Axiom 

|caucus| > 2 * |faulty_members(caucus)| A ( V p : ok(p) A p £ caucus D v(p) = t) 

D maj(caucus, v) = t 

The function application faulty. members(caucus) that appears here is the set of 
faulty (i.e., not ok) processors in the set caucus: 

faulty .members: function[set — ► set] = ( A m\ : ( X z : z £ mi A ->ok(z))) 

Vertical bars denote the cardinality function. The only properties we require of this 
function are captured in the following axioms. 

| * 1|: functioned — » nat] 

non-empty .ax: Axiom (3p:p£mi)<=> |mi | ^ 0 

card.remove.ax: Axiom z 6 mi D |mi — {z} | = |mi| — 1 

A second requirement on the maj function is that its value depends on only 
those elements of the vector corresponding to members of the set caucus. 

maj_ext: Axiom 

(Vp : p £ caucus D »u(p) = u 2 (p)) D maj(caucus, i>i) = maj(caucus, v 2 ) 

We now return to the specification of OMIC. The two behaviors that were stated 
above (for the cases m = 0, and m > 0, respectively) could be specified as axioms 
defining the function; we prefer, however, to specify the function definitionally and 
to deduce those properties as (straightforward) lemmas. The advantage of the defi- 
nitional specification is that the Eh dm typechecker will guarantee its soundness (in 
the sense of not introducing inconsistencies). To do this, we are required to exhibit 
a measure function that takes the same arguments as OMIC and whose value is a 
natural number that can be proved to decrease across recursive calls. In the present 
case, we use the measure function terminates that simply returns its first argument 
(i.e., the number of rounds). The final specification for OMIC is given in Figure 4.1. 






terminates: function[rounds, vector, set — » nat] = ( A r, v, caucus — ► nat : r) 

OMIC: Recursive function[rounds, vector, set — ► function [processors — ► vector]] 
( A r, r, caucus : 
if r = 0 
then (A p : 

(Ag : 

if p G caucus A q E caucus 
then send(r, v(q)^q^p) 
else undef 
end if)) 
else (A p : 

{Xq-.' 

if p 6 caucus A g 6 caucus 
then if p = q 

then send(r, v(q) y q,q) 
else maj(caucus - { q }, 

OMIC(r — 1, distr(r, v(q ), g), caucus — {#})(/>)) 

end if 
else undef 
end if)) 

end if) 

by terminates 


Figure 4.1: Our specification of the Oral Messages Algorithm 
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We invite the reader to compare this specification with that of Bevier and Young 
that was shown in Figure 3.1. Since our specification is pretty-printed (a function 
performed automatically by Ehdm), while Bevier and Young’s is given in raw text 
form, the versions shown in Appendix A, which reproduce the exact text submitted 
to their respective theorem proving environments, allow more exact comparison. 

The Interactive Consistency conditions IC1 and IC2 are easily stated as theorems 
to be proven: 

Cl-final: Theorem 

ok(p) Aok(g) D OMIC(m, v,fullset)(p)(y) = OMIC(m, v,fullset)(g)(y) 

C2_final: Theorem ok(p) A ok(g) D OMI C(m, v, fullset)(p)(g) = v(q ) 

where fullset is the set of all processors. 

As in the informal proof of Section 2.2.1, we begin by proving a lemma similar 
to IC2. The proof is by induction and in formal verifications it is usually convenient 
to reformulate the theorem to be proved as a predicate on the induction variable. 
Here, we call the predicate C2prop. 

C2prop: function[rounds — > bool] = 

(A r : (Vp,g, caucus, v : 
ok(p) A ok(g) 

Ap€ caucus A q € caucus 

A |caucus| > 2 * |faulty_members(caucus)| + r 
D OMIC(r, t>, caucus)(p)(g) = u(g))) 

The base case of the induction (i.e., C2prop{ 0)) follows by straightforward appli- 
cation of definitions; the inductive step (i.e., r < m A C2prop{r) D C2prop(r + 1)) 
follows from two lemmas. The first, which asserts that a good processor has the 
correct opinion of its own value, is straightforward: 

ok-self: Lemma ok(y) A y € caucus D OMIC(r, v 2 ,caucus)(y)(y) = v 2 (y) 

The second, which asserts that under certain conditions a good processor forms the 
correct opinion of the private value of another good processor, is more complex. 

ok.others: Lemma 

r < m A |caucus - {q}\ > 2 * |faulty _members(caucus - {g})| 

A ok(y) A ok (q) 

A y 6 caucus A q £ caucus 
A y ^ 9 

A ( V z, Vj : z G caucus A ok(z) A z ^ q 

D OMIC(r, t>x, caucus - {9})(y)(^) = ui(*)) 

D OMIC(r + 1, t> 2 , caucus)(y)( 9 ) = v 2 {q) 
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Verification of this property depends on the majax axiom of the maj function. 

The two lemmas above are sufficient to establish the inductive step for verifi- 
cation of C2prop[r ); observe that the hypothesis to the inductive step discharges 
the quantified subexpression in ok.others. The theorem C2-final follows straightfor- 
wardly from C2prop(r ) by substitution of m for r and fullset for caucus , and using 
the axiom 

fullset_card_ax: Axiom [fullset | — n A |faulty_members(fullset)| < m 
and the constraint that less than a third of the processors may be faulty: 

mn.prop: Formula 3 * m < n 

This property is stated as a formula in the assuming section of the Ehdm module 
that specifies the theory developed here. It specifies an assumption on the param- 
eters m and n to the module: inside the module, this assumption is treated as an 
axiom; it must be discharged whenever the module is instantiated. 

IC1 is similarly proved by induction, using the following predicate. 

Cl prop: function [rounds — ► bool] = 

( \ r : ( V p, g, y, caucus, v : 
ok(p) A ok(g) 

A p G caucus A g £ caucus A y € caucus 

A |caucus| > 3*r Ar > |faulty_members(caucus)| 

D OMIC(r, v, caucus)(p)(y) = OMIC(r, v f caucus)(g)(y))) 

Again the base case is straightforward; the inductive step has two cases, depending 
on whether the processor y is faulty or not. The case that it is faulty is dealt with 
in the following lemma, whose proof is a consequence of the maj-ext axiom of the 
maj function. 

agree jiok: Lemma 

t < m A [caucus) >3*(r+l)Ar+l> [faulty _members(caucus)| 

A ok(p) A ok(g) 

A p £ caucus A g £ caucus A y £ caucus 
A ~»ok(y) 

A ( V z, v\ : z £ caucus — {y} 

D OMIC(r, v\, caucus — {y})(p)(z) = OMIC(r, v\ ) caucus - {y})(g)(^)) 
D OMIC(r -f 1, v 2 , caucus)(p)(y) = OMIC(r + 1, u 2 , caucus) (g)(y) 
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The case when y is nonfaulty is treated in the following lemma 
agree_ok: Lemma 

r < m A |caucus| >3*(r+l)Ar+l> |faultyjnembers(caucus)| 

A ok(p) A ok(g) 

A p € caucus A q € caucus Ay 6 caucus 
A ok(y) 

3 OMIC(r + 1, v 2 , caucus)(p)(j/) = OMIC(r + 1, V 2 , caucus) (y)(y) 

whose proof is a consequence of C2 Jinal. 

These two lemmas are sufficient to establish the inductive step for Cl. final; 
note that the hypothesis to this step discharges the quantified subexpression in 
agree.nok. Cl.final follows from Clprop(r ) in the same way that C2.final follows 
from C2prop(r). 

The full specification and verification requires development of some “background 
knowledge.” For example, the inductions require a specialized induction scheme that 
goes from 0 only as far as m. This is stated as the Lemma round.induct that is 
ultimately derived from an axiom for Noetherian induction contained in a standard 
EHDM library module. The variable round.prop is some arbitrary property of rounds 
that is to be shown to hold for all rounds. 

round_prop: Var function[rounds — ► bool] 
round.induct: Lemma 

(round.prop(O) A ( V r : r < m A round.prop(r) 3 round_prop(r + 1))) 

3 round.prop(s) 

A full listing of the formal specification is provided in Appendix C, together 
with Ehdm’s “proof chain” analysis for Cl Jinal. The latter identifies the axiomatic 
foundation for our development: this comprises the 6 axioms and the assumption 
shown here, plus an axiom for induction and another for function extensionality 
that come from library modules. The subsidiary lemmas required to carry out the 
formal verification number 23 (plus the two theorems), with another 4 in library 
modules, and a further 20 typecheck correctness conditions (tccs — these are proof 
obligations that must be discharged to ensure type-correctness) that are generated 
by the typechecker. Only 2 of the tccs require user-generated proofs; the other 18 
are proved automatically. 




Chapter 5 


Discussion and Conclusion 


5.1 Discussion 

We have presented the formal specification and verification of an algorithm for In- 
teractive Consistency derived from the Oral Messages algorithm for the Byzantine 
Generals Problem. Both the specification of the algorithm and the arguments for 
its correctness are straightforward and closely modeled on those given by Lamport, 
Shostak and Pease in their journal presentation [9]. Development of the formal spec- 
ification and its verification in Ehdm took about four days. By comparison, Bevier 
and Young [3], using the Boyer-Moore theorem prover, found formal verification of 
their version of the algorithm “a fairly difficult exercise in mechanical theorem prov- 
ing” that occupied them for about a month. We do not know all the complexities 
that confronted Bevier and Young, and so we cannot identify, much less apportion 
credit to, all the reasons why we apparently found the verification easier than them. 

However, one explanation for these different assessments of the difficulty of the 
exercise may lie in the different formulations employed for the algorithm. Bevier and 
Young used the Byzantine Generals formulation, which must be applied iteratively 
in order to solve the Interactive Consistency problem that is the topic of real interest, 
and whose recursive subcase likewise requires iteration. This potentially complicates 
the inductions at the heart of the proof (since the recursive subcase is not simply 
a smaller instance of the original problem), and the larger verification along with 
it. The specification of the algorithm may become similarly complicated in this 
formulation. In contrast, our reformulation of the Oral Messages algorithm solves 
the Interactive Consistency problem directly, and its recursive subcase is a smaller 
instance of itself. The formal specification, main inductions, and overall verification 
are then entirely straightforward. 

The lesson here is a variation on the well-known observation that it is sometimes 
easier to prove a stronger than a weaker theorem when using induction. In partic- 
ular, it is much easier to prove properties of an algorithm whose recursive subcases 
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are exact replicas of itself: and it may be worth modifying the algorithm, or its 
requirement, or both, in order to make this so. A related observation, one that we 
first heard explicitly articulated by our colleague Shankar, is that recursions should 
always be formulated so that the base case is completely different from the recursive 
case — since otherwise one may end up verifying substantially the same argument 
twice. 

But our simpler verification cannot be entirely attributed to our reformulation 
of the Oral Messages algorithm into the IC form, for we have also verified the BG 
version of the algorithm as considered by Bevier and Young. Our specification 
of the BG form is not exactly the same as theirs, since our richer specification 
language allows us to specify the algorithm without the need to simulate a pair of 
mutually recursive functions (see Appendix B). Nonetheless, our BG formulation is 
substantially the same as Bevier and Young’s and yet its verification is only a little 
more complex than that of OMIC. 1 However, we must admit that the formulation 
and verification of the BG version would have been significantly more difficult had 
we not already performed the IC version; that is specification and verification of IC 
followed by BG is probably much simpler than tackling BG alone. 

But allowing for the advantage we gained by choosing the more tractable ap- 
proach, we still seem to have found this exercise more straightforward that Bevier 
and Young, and we attribute some of this to the design decisions embodied in Ehdm. 
The specification language of Ehdm is intended to provide a fairly direct and natural 
means for expressing a variety of mathematical concepts, while retaining a straight- 
forward logical foundation. We were gratified to find that the language helped us 
to achieve clear descriptions of these tricky algorithms. We find the strong type 
system and higher-order capabilities particularly helpful in this regard. Identify- 
ing the types of the variables and functions involved is a valuable first step in the 
formulating the specification, since it suggests the ways in which functions should 
be combined and thereby, in this case, helps determine the shape of the recursion. 
Higher-order logic allows many ideas to be expressed is a direct manner: thus, we do 
not require the mutual recursion that complicates Bevier and Young’s specification, 
and we can represent values as functions, without the need to introduce lists. 

We have had similar experiences with other specifications that we have under- 
taken. For example, our formal development in Ehdm of a model for fault-masking 
and transient-recovery in digital flight-control systems [13, 14] was undertaken in 
parallel with a similarly detailed development using conventional pencil-and-paper 
mathematical notation [5,6]. The Ehdm version took no longer to develop than the 
other, is more general, is equally readable, and has been fully verified. 

The simplifying reformulation of the Oral Messages Algorithm into its IC form 
is very much the kind of benefit that we strive to obtain from formal methods (see, 

1 The verification, which is available from the author on request, was obtained by modifying the 
OMIC version, and took about a man-day to produce. 
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for example, our improved argument for the correctness of the Interactive Conver- 
gence clock synchronization algorithm [15,16]). We are strongly of the opinion that 
formal methods must contribute to, and cannot stand apart from, established and 
informal practices in software and hardware engineering. Thus, specifications must 
be readable by others than their authors, and formal verifications must yield a chain 
of argument that can be presented to, and will convince, a suitably knowledgeable 
human reviewer. 

5.2 Conclusion 

As with other formal developments that we have performed, we derived a significant 
benefit from this exercise quite apart from the mechanically- checked verification of 
an interesting argument. Here, the benefit was a reformulation of the Oral Messages 
Algorithm to solve the Interactive Consistency, rather than the Byzantine Generals 
problem. This is not only a more useful form of the algorithm in practice, it is rather 
simpler to specify and to verify. As always, this benefit could have been obtained 
without formalization, but it was the discipline of formalization that led us to focus 
on the problem in the manner required. 

The simplification produced by our reformulation can be gauged by comparing 
our formal specification and verification with that of Bevier and Young. Bevier and 
Young state [3, page 1] 

• “We believe that our formulation provides a very clear and unam- 
biguous characterization of the algorithm. 

• “Our machine checked proof elucidates several issues which are 
treated rather lightly in the published version of the proof. In par- 
ticular, the invariant maintained in the recursive subcases of the 
algorithm is significantly more complicated than is suggested by 
the published proof. 

“The latter two advantages arise as a consequence of providing a fully 
formal proof, whether machine checked or not. However, the use of a 
powerful mechanical theorem prover as a checker is a boon in managing 
the complexity of the formal proof.” 

Regarding the first of these claims, we believe that our formulation is rather clearer 
and simpler (and more useful) than Bevier and Young’s, but that may be a matter 
of taste. We believe we have demonstrated that their second claim is mistaken: our 
machine-checked proof is essentially the same as the published version of the proof, 
and we think it likely that the complexity discovered by Bevier and Young was an 
artifact of their formalization and of the theorem prover at their disposal. Rather 
than agreeing that “a powerful mechanical theorem prover . . .is a boon in managing 
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the complexity of the formal proof,” we believe that a mechanical theorem prover 
should help the user develop reasonably clear and straightforward proofs. 

In summary, we believe this small example provides some substantiation for our 
belief that the benefits of formal specification and verification are best assisted by 
rather rich specification languages that permit natural forms of expression, and by 
approaches to theorem proving that permit fairly direct control by the user. 

Finally, recall that in the Introduction, we stated that our other motivations 
for undertaking this work were to see whether we derived benefits similar to those 
obtained in earlier formal verifications that we had undertaken (e.g, discovery of 
flaws in previous proofs), and to compare the difficulty of verifying of an algorithm 
for interactive consistency with that of one for clock synchronization. 

Unlike our experience with clock synchronization algorithms, the benefits we 
derived from formal verification of OMIC did not include discovery of flaws in the 
previously published proof of correctness of the algorithm. However, building on 
the experience gained in the exercise described here, we have discovered an error 
in the algorithm (not just the proof) of Thambidurai and Park [21] for Interactive 
Consistency under a hybrid fault model; working with Patrick Lincoln, we have also 
developed and formally verified a correct algorithm for this problem. 

Regarding difficulty, we can report that we found verification of OMIC to be an 
order of magnitude simpler than that of Interactive Convergence [15,16]: four days 
work compared to about 40, and 23 subsidiary lemmas compared with nearly 200. 
Given its relatively small size, but rather interesting character, we invite others to 
try formal specification and verification of the Oral Messages Algorithm using their 
favorite verification system. We have used the Byzantine Generals formulation of 
this algorithm as one of the test cases in the development of our new Prototype 
Verification System, PVS [11]. Using PVS, we are now able to construct the main 
proofs for correctness of OMBG in under an hour. For those more interested in 
fault-tolerant algorithms than the performance-testing of verification systems, an 
interesting challenge is to develop and formally verify some of the many variants 
that have been proposed for the Oral Messages Algorithm. We, for example, have 
investigated the algorithm of Thambidurai and Park as already mentioned, and have 
also formally verified a generalization of the algorithm used to provide Interactive 
Consistency in the Draper FTP architecture [7]. 
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Appendix A 

The “Real” Specifications 


In this Appendix we reproduce the “real” specifications of the algorithms employed 
by Bevier and Young and by ourselves. Bevier and Young’s specification differs 
from that of Figure 3.1 by combining the pair of mutually recursive functions into 
a single function with a “flag” argument; our specification is the same as that given 
on page 17, but is reproduced here in its raw text form. 


Definition 

(vom fig m g v 1 vec) 

(if fig 

(if (zerop m) 

(vomO g v 1 vec) 

(votelist 

(pair (vomO g v 1 vec) 

(vom f (subl m) 1 (vomO g v 1 vec) 1 vec) 

1 ))) 

(if (listp g-list) 

(pair (vom t m (car g) (get (car g) vomO) 

(delete (car g) 1) vec) 

(vom f m (cdr g) v 1 vec) 

(delete (car g) 1)) 

(init nil (length vec)))) 


Figure A.l: Bevier and Young’s Specification — The Real Version 
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OMIC: RECURSIVE function [rounds , vector, set 

-> function [processors -> vector]] = 
(LAMBDA r, v, caucus : 

IF r = 0 

THEN (LAMBDA p : 

(LAMBDA q : 

IF member (p, caucus) AND member(q, caucus) 

THEN send(r, v(q) , q, p) ELSE undef END IF)) 
ELSE (LAMBDA p : 

(LAMBDA q : 

IF member (p, caucus) AND member(q, caucus) 

THEN IF p = q 

THEN send(r, v(q), q, q) 

ELSE maj (remove (caucus, q) , 

0MIC(r - 1, distr(r, v(q) , q) , 
remove (caucus, q))(p)) 

END IF 
ELSE undef 
END IF)) 

END IF) 

BY terminates 


Figure A. 2: Our Specification — The Raw Text Version 



Appendix B 

The Byzantine Generals 
Formulation of the Algorithm 


We specify the Byzantine Generals formulation of the Oral Messages algorithm as 
a function OMBG of four arguments: G the identity of the General, m the number 
of rounds, / the value the General wishes to communicate, and caucus , the set of 
participants (which includes the General). OMBG will return a vector of values 
in which OMBG(G, m,C caucu$)(p) is lieutenant p’s opinion of the General’s value. 
The correctness conditions are the following. 


BG1 Jinal: Theorem ok(p) A ok (q) 

D OMBG(G, m, C fullset)(p) = OMBG(G,m f *,fullset)(g) 
BG2_final: Theorem ok(p) A ok(G) D OMBG(G, m, t t fullset)(p) = t 


The specification of OMBG is rather interesting; it is due to our colleague Shankar. 
In the case r = 0, lieutenant p’s component of the vector returned is simply the value 
received by p from the General; in the case r > 0, lieutenant p’s component of the 
vector is the value the General receives from himself when p = G, otherwise it is the 
result of applying the maj function to the vector of values that p obtains when each 
of the lieutenants in the caucus (less G but including p himself) acts as the General 
in the OMBG algorithm with r - 1 rounds to distribute the value received by that 
lieutenant from the original General. Notice how the higher-order capabilities of 
the Eh DM specification language allow us to specify the inner, iterative application 
of OMBG by means of a A-abstraction, thereby avoiding the mutually recursive 
functions of Bevier and Young’s specification. 

The formal verification of OMBG is very similar to that of OMIC, and was 
derived from that of OMIC in less than a day. It is available from the author on 
request. 
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Appendix B. The Byzantine Generals Formulation of the Algorithm 


terminatesBG: function[processors, rounds, T, set — ► nat] == 

( A p, r, t , caucus — ► nat : r) 

OMBG: Recursive functionfprocessors, rounds, T, set vector] = 

(A (7,7", caucus : 
if r = 0 

then (A p : if caucus(p) A caucus(G) 
then send (r, Z,G,p) 
else undef 
end if) 

else (A p : if caucus(p) A caucus (G) 
then if p — G 
then send(r, t,G,G) 
else maj(caucus — {G}, 

( A q : OMBG(g, r - 1, send(r, t, G, g), caucus - {G})(p))) 

end if 
else undef 
end if ) 

end if) by terminatesBG 


Figure B.l: Our Formulation of the Byzantine Generals Version of the Algorithm 


Appendix C 

The Full Specification and 
Verification 


The specification text and verification output that follow were processed by Eh dm 
Version 5.2; some minor changes to the input syntax (e.g., deletion of the PROOF 
keyword) will be necessary in order to use the current version of Ehdm (numbered 
6 . 1 . 1 ). 

C.l The Specification 

We reproduce here the text of our specification and verification for the IC version 
of the OM algorithm. The text comprises the module consensus. In the inter- 
ests of brevity, we do not reproduce the system-generated module consensus jtcc 
that contains the “typecheck-consistency conditions” (tccs), nor the module top 
that gives their proofs. Neither do we reproduce the library modules noetherian, 
induction and functionprops. The module noetherian specifies the axiom of 
Noetherian Induction (see, for example [15, page 99], [13, page 62] or [17, pages 
57-61]), and the module induction (see, for example [13, page 63]) derives some 
more specialized induction schemes from that general formulation. One of these 
is used to prove the round_induct induction scheme over rounds that is employed 
here. The functionprops module (see, for example [15, page 99]) simply -specifies 
an axiom of function extensionality. 

C.1.1 Module “Consensus” 

This module contains the specification and verification of the OMIC algorithm. Cer- 
tain subsidiary concepts, such as sets and cardinality are defined here, too. Normally 
these concepts are imported from library modules (see, for example [13, page 66]), 
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Appendix C. The Full Specification and Verification 


but so few of their properties are needed here that we have preferred to specify them 
in line. 

consensus: Module [m,n: nat] 

Exporting all 
Assuming 

mn_prop: Formula 3 * m < n 

Theory 

x : Var nat 

processors: Type from nat with ( A x : x < n) 
rounds: Type from nat with ( A x : x < m) 

T: Type 

vector: Type is function[processors — ► T] 

r, s : Var rounds 

v,Vi,V 2 ’ Var vector 

p, q, y, z: Var processors 

undef: T 

t : Var T 

set: Type is function [processors — ♦ bool] 
fullset: set == (A z : true) 
ok: function [processors — ► bool] 
caucus, 77i i , m 2 : Var set 

p G mi: function [processors, set — ► bool] == (Ap, mi : mi(p)) 

faulty .members: functionfset — *■ set] = ( A mi : ( A z : z E mi A ->ok(z))) 

★1 — {*2}: functionfset, processors — ► set] == 

( A caucus, q : caucus with [( q ) := false]) 

|*1|: functionfset — ► nat] 

non-empty _ax: Axiom (3 p : p E mi) O |mi | / 0 
fullset_card_ax: Axiom |fullset| = nA |faulty_members(fullset)| < m 
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alljok: Lemma 0 = (faulty _members(caucus)| A p 6 caucus 3 ok(p) 

card_remove_ax: Axiom z E m x 3 |m x — {z}| — |mi| — 1 

faulty_members_card_remove_ok: Lemma 
z Errix A ok(z) 3 |faultyjnembers(mi - {*})[ = |faultyjnernbers(mi)| 

faulty _members_card.remove_nok : Lemma 

z € m x A -iok(z) 3 (faulty _members( 77^1 - {z})\ = |faultyjnembers(m 1 )| - 1 

maj: function [set, vector — ► T ] 

majax: Axiom |caucus| > 2 * (faulty _members(caucus)| 

A ( V p : ok(p) Ap E caucus 3 v(p) = t) 

3 maj(caucus, u) = t 

majjext: Axiom ( Vp : p E caucus 3 t^(p) = ii 2 (p)) 

D maj(caucus, t»i) = maj (caucus, i; 2 ) 

send: function [rounds, T, processors , processors — ► T] 

send.ax: Axiom ok (p) A ok (q) 3 send(r, t , q y p) = t 

distr: functionfrounds, T, processors — ► vector] == 

(A r,t,p : ( A z : send(r, t,p, z))) 

terminates: functionfrounds, vector, set — ► nat] == (Ar,v, caucus — ► nat : r) 

OMIC: Recursive functionfrounds, vector, set — ► function [processors -+ vector]] 

= ( A r, v y caucus : 
if r = 0 

then ( A p : ( A q : 

if p E caucus A q £ caucus then send(r, v(q), q,p) else undef end if)) 
else ( A p : ( A q : 

if p E caucus A q £ caucus 
then if p — q 

then send(r, i ;(<?), q , q ) 
else maj(caucus — {g}, 

OMIC(r- 1, distr(r, v(q), q), caucus — {<?})(p)) 

end if 
else undef 
end if)) 

end if) by terminates 

Cl -final: Theorem ok(p) A ok(</) 

3 OMIC(TO,v f fullset)(p)(y) = OMIC(m, u,fullset)($)(y) 

C2 -final: Theorem ok(p) A ok(q) 3 OMIC(m, v, fullset)(p)(g) = v(q) 
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Clprop: function[rounds — ► bool] = 

( A r : ( V p, g, y, caucus, v : 

ok(p) A ok(g) A p £ caucus 
A q G caucus 

A y G caucus A | caucus | > 3 * r A r > (faulty _members(caucus)[ 
3 OMIC(r, u,caucus)(p)(y) = OMIC(r, v, caucus)(g)(y))) 

C2prop: function[rounds — ► bool] = 

(A r : (Vp, g, caucus, v : 

ok(p) A ok (g) A p (E caucus 

A g G caucus A (caucus) > 2 * (faulty _members(caucus)| -I- r 
3 OMIC(r, v, caucus)(p)(g) = u(g))) 

Cl: Lemma Clprop(r) 

C2: Lemma C2prop(r) 

Proof 

Using induction, functionprops[processors, bool] 
i : Var nat 

round_prop: Var function[rounds — ► bool] 

round induct: Lemma (round_prop(0) 

A(Vr:r<mA round.prop(r) 3 round_prop(r + 1))) 

3 round_prop(s) 

roundinduct_proof: Prove 

roundinduct {r <— if z@pl in rounds then i@pl else 0 end if} from 
limited-induction 
{m <- 0, 

77li <— 771 , 

p +— ( A i : if i in rounds then round_prop(z) else false end if), 
n s} 

distr_prop: Lemma ok(p) A ok(g) 3 distr (r, v(p) } p)(q) = v(p) 

distr_prop_proof: Prove distr.prop from 
send. ax {t <- v{p), p *- q, q <— p} 

OMO.prop: Lemma OMIC(0, v, caucus)(p)(g) 

= if p 6 caucus Aq E caucus then send(0,t;(g) 1 9,p) else undef end if 

OMO-prop.proof: Prove OMO.prop from OMIC {r <— 0} 
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OM_prop: Lemma r > 0 D OMIC(r, v , caucus) (p)(</) 

= if p E caucus A q £ caucus 
then if p = q 

then send(r, v(q ), q ) q) 

else maj(caucus — {<j} 5 OMIC(r — l,distr(r, v(q), q) 7 caucus — {<?})(p)) 
end if 
else undef 
end if 

OM .prop .proof: Prove OM.prop from OMIC 

OMO-ok: Lemma ok(p) A ok(q) A p £ caucus A q £ caucus 
D OMIC(0, caucus)(p)(g) = v(q) 

OMO_ok_proof: Prove OMO_ok from OMO.prop, send.ax {r 4— 0, t <— u(<?@c)} 

okjself: Lemma ok(y) Ay £ caucus D OMIC(r, v 2 , caucus) (t/)(y) = v 2 (y) 

ok-selLproof: Prove okjself from 
OM-prop {v «- v 2 , P <- y, q «- y}, 

OMO.prop {v *-v 2 , p<-y, q <-y), 
send.ax {p *- y, q *- y, t *- u 2 (y)} 

'"K; 

remove-ok -member: Lemma 

z £ mi A ok(z) D (p £ faulty _members(mx — {z}) O- p £ faulty_members(mi)) 

remove_ok-member_proof: Prove remove-ok_member from 
faulty -members {z «— p), faulty .members {mi «— m x — {2}, 2 +— p} 

remove_ok: Lemma 2 £ mi A ok(2) D faulty _members(mi — {2}) — faulty_members(mi) 

remove_ok_proof: Prove remove_ok from 
remove -ok .member {p a@p2}, 

extensionality {F faulty_members(mi), G <— faulty _members( mi — {2})} 
remove-nok .member: Lemma 

2 £ mi A — «ok(2) D (p £ faulty _members(mi — {2}) p £ faulty_members(7Tii) — {2}) 

remove_nok_member_proof: Prove remove_nok_member from 

faulty .members {z p} } faulty .members {mi mi — {2}, 2 p} 

remove-nok: Lemma 2 £ A -»ok(z) 

D faulty jnembers(mi — {2}) = faulty_members(mi) - {2} 

removem ok .proof: Prove remove _nok from 
removemok-member {p <— a@p2}, 

extensionality {F faulty_members(m.i) — {2}, G faulty _members(mi — {2})} 

faulty_members_card_remove_ok_proof: Prove faulty .members _card_remove_ok from 
remove.ok 
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faulty _members_card_remove_nok_proof: Prove faulty miernbers.cardjrernovejiok from 
remove_nok, faulty .members, card_remove_ax {mi faulty_members(mi@c)} 

ok .card jemove: Lemma 
r < m A q £ caucus A 6k(q) 

D |caucus| > 2 * (faulty _members(caucus)| + 1 

D (caucus — {q} \ > 2 * |faulty.members( caucus — {g})| + r 

ok .card jem ovenproof: Prove ok .card .remove from 
card .remove _ax {mi caucus, z g}, 

faulty _mernbers.card_remove_ok {mi caucus, z *— q] 

ok.others: Lemma r < m 

A (caucus - {q} \ > 2 * (faulty _members(caucus - {q})\ 

A ok(y) A ok(q) 

Ay £ caucus 
A q E caucus 
Ay t* Q 
A ( V z,vi : 

z £ caucus A ok(z) A z ^ q 

D OMIC (r, ui, caucus - {g})(y)(z) = vi(*)) 

D OMIC(r + 1, f 2 , caucus )(y)(g) = v 2 (q) 

next_round: function[rounds — * rounds] == 

( A r — ► rounds : if r < m then r + 1 else 0 end if) 

ok-others.proof: Prove 

ok.others {z p@pl, v x <— distr(next jound(r), v 2 (g), ?)} from 
majax 

{caucus caucus - {y}, 

v OMIC(r, distr(next_round(r), v 2 (q) y y), caucus — {y})(y), 

t<-v2 (g)}, 

OM.prop {r next.round(r), v «— u 2 , p <— y}, 
distr.prop {r <— next _round(r), v *— v 2l p <— y, q <— y}, 

distr.prop {r <— next_round(r), d+-v 2 , p <— y, q *— y), 

distr.prop {r next jound(r), v +— v 2 , p <— y, y <— p@pl} 

C2prop_0: Lemma C2prop(0) 

C2prop_0 .proof: Prove C2prop_0 from 
C2prop {r <— 0}, 

OMO.ok {p <— p@pl, q q@pl } v <— v@pl i caucus 4— caucus@pl] 

C2prop_r: Lemma r < m A C2prop(r) D C2prop(r + 1) 
remove.others: Lemma p £ caucus Ap/gDpG caucus - {y} 
remove.others.proof: Prove remove.others 
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C2prop_r -proof: Prove C2prop_r from 
C2prop 
{v <— 

q z@p3, 

p +— p@p2, 

caucus caucus@p 2 - {<j@p 2 }}, 

C2prop {r <— next_round(r)}, 
ok.others 
{? <- q@p2, 
y <- p@p2, 
v 2 v@p2 ) 
caucus <— caucus@p 2 }, 
okjself 

{r <— next_round(r), 
y <- p@p 2 , 

v 2 v @p2, 

caucus <— caucus@p 2 }, 

ok_card_remove {caucus «— caucus@p 2 , q <— <jr@p 2 }, 
remove-others {caucus <— caucus@p 2 , q +— g@p 2 , p 4 — p@p 2 }, 
remove-others {caucus 4 — caucus@p 2 , q 4 — q@p2 } p 4 — z@p3} 


C2_proof: Prove C2 from 
round-induct {round.prop 4 — C2prop, 5 +-r}, 
C2prop_0, 

C2prop_r {r<- r@pl} 


agree_nok: Lemma r < m 

A |caucus| > 3 * (r + 1) 

A r + 1 > (faulty -members(caucus) | 

A ok(p) A ok(q) 

ApE caucus 
A q G caucus 
A y G caucus 
A ->ok(t/) 

A ( V z, t>i : 

z £ caucus — { 1 /} 

D OMIC(r, t;i , caucus — {y})(p)(z) 

= OMIC(r, vi, caucus - {y})(g)(z)) 
D OMIC(r+ 1, i> 2 , caucus)(p)(p) = OMIC(r+ 1, v 2 , caucus)(g)(j/) 
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agree-n ok -proof: Prove 

agree _n ok {z <— p@p 3 , v\ *- distr(next-round(r ) 3 t; 2 (y), y)} from 
OM.prop {r 4— next_round(r), v 4— v 2 , q +— y}, 

OM-prop {r 4- next_round(r), v 4— v 2> Q *— y, V «“ q), 
majjsxt 

{caucus 4— caucus — {y}, 

Vi 4- OMIC(r, distr(next_round(r),i; 2 (y),y ) 3 caucus - {y})(p), 
V2 4— OMIC(r, distr(next_round(r) , i> 2 (y), y), caucus - (y})(g)}, 
distr.prop {r 4— next_round(r), v 4— u 2) p 4— y } 3 

distr.prop {r 4 — next_round(r), v +- t>2, P V, q 2/}, 

distr.prop {r 4 — next_round(r), v 4 — v 2 i P <— S/i <— P@pl} 


agree-ok: Lemma r < m 

A |caucus| > 3 * (r + 1 ) 

A r + 1 > |faulty_members(caucus)| 

A ok(p) A 6 k(q) A p 6 caucus A q E caucus A y € caucus A ok(y) 
D OMIC(r+ l,t> 2 , caucus) (p)(y) — OMIC(r + 1 , u 2 , caucus)(g)(y) 


agree.ok .proof: Prove agree _ok from 
C 2 {r 4— next_round(r)}, 

C 2 prop {r 4 — next-round (r), q +- y, v 4— ^2} , 
C 2 prop {r 4— next-round (r), p <— q, q *— y, ^ti 2 } 


all j^k .proof: Prove all.ok from 

non-empty _ax {mi +— faulty-members(caucus)}, 
faulty_members {mi 4— caucus, z 4— p} 


Cl prop- 0 : Lemma Clprop(O) 


Clprop.O -proof: Prove Clprop.O from 
Clprop {r 4 — 0}, 

OMO-ok {p 4 — p@pl, q 4 — y@pl, v 4 — v@pl, caucus 4 — caucus@pl], 
OMO.ok {p 4— q@pl, q 4 — y@pl, v 4 — v@pl, caucus 4 — caucus@pl], 
alLok {p 4— y@pl, caucus 4— caucus@pl), 
nat Jnvariant {nat.var 4 — | faulty jmembers(caucus@pl)|) 


Clprop_r: Lemma r < m A Clprop(r) D Clprop(r + 1 ) 
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Clprop_r_proof: Prove Clprop_r from 
Cl prop 

{v 4 — ui@p3, 

y <— ^@p3, 

p «— p@p2, 

q 4— 

caucus *— caucus@p2 — {p@p2}}, 

Clprop {r <— next_round(r)}, 
agree_nok 
{v 2 i— v@p2 y 
caucus caucus@p2, 

p <- p@p2, 

q <- q@p2, 
y <- y@p2], 
agree_ok 

{t) 2 4 — v@p2, 
caucus caucus@p2, 

P <- P%>2, 
q 4- g@p2, 
v <- y@p2}, 

remove-others {p p@p2, g <— t/@p2, caucus >— caucus@p2}, 
remove.others {p *— <?@p2, q 4— y@p2, caucus <— caucus@p2}, 
card .remove _ax {mi <— caucus@p2, z +- p@p2}, 
faulty-members-card_remove_nok {mi 4— caucus@p2, z <— y@p 2 } 

Cl -proof: Prove Cl from 
roundinduct {round-prop +— Clprop, s <— r}, 

Clprop.O, 

Clprop_r {r <— r@pl} 

Cl -final -proof: Prove Cl_final from 

Cl {r 4- m}, Clprop {r 4— m, caucus 4— fullset}, fullset-card.ax, mn-prop 
C 2 _final_proof: Prove C 2 _final from 

C 2 {r 4— m}, C 2 prop {r *— m, caucus fullset}, fullset _card_ax, mn_prop 
End consensus 
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C.2 Proof-Chain Analysis 

The following pages reproduce the output from the Eh dm proof-chain analyzer in 
“terse mode” applied to the formula Cljfinal in module consensus. The analysis 
for C2_f inal is similar. The Ehdm proof-chain analyzer examines the macroscopic 
structure of a verification — checking that all the premises used in a proof are either 
axioms, definitions, or formulas which are, themselves, the target of a successful 
proof elsewhere in the verification. If any formulas are used from a module having 
an assuming clause, then the proof-chain analyzer checks that those assumptions 
are discharged by successful proofs; similarly, if formulas are used from a module 
having a tcc module, then the proof-chain analyzer checks that all the tccs in 
that module are discharged by successful proofs. The proof-chain analyzer ignores 
unsuccessful proofs (such as automatically-generated TCC proofs) when a successful 
proof for the same formula can be found. The “terse mode” output reproduced here 
provides a commentary on only the “interesting” cases, namely proof obligations 
involving assuming clauses and tccs, and a summary. All the proofs listed in the 
summary were performed by the Ehdm theorem prover in “checking mode.” 

Terse proof chain for formula Cl_final in module consensus 
Interesting cases from the analysis follow; see summary for status 

Use of the formula 

consensus [EXPR, EXPR] .Cl_final 
requires the following TCCs to be proven 
consensus_tcc [EXPR, EXPR] .processors_TCCl 
consensus_tcc [EXPR, EXPR] .rounds_TCCl 
consensus_tcc [EXPR, EXPR] . 0MJTCC1 
consensus_tcc [EXPR , EXPR] . 0M_TCC2 
consensus_tcc [EXPR, EXPR] . Ci_f inal_TCCl 
cons ensus_tcc [EXPR, EXPR] .round_induct_TCCl 
consensus_tcc [EXPR, EXPR] . round_induct_TCC2 
consensus_tcc [EXPR, EXPR] . round_induct_proof_TCCl 
consensus_tcc [EXPR , EXPR] , round_induct_proof_TCC2 
consensus_tcc [EXPR , EXPR] . 0M0_prop_TCCi 
consensus_tcc [EXPR, EXPR] , OM_prop_TCCl 
consensus_tcc[EXPR, EXPR] . 0M0_ok_TCCi 
consensus_tcc [EXPR, EXPR] . ok_others_TCCl 
consensus_tcc [EXPR, EXPR] .next_round_TCCl 
consensus_tcc [EXPR, EXPR] . C2prop_r_TCCl 
consensus_tcc [EXPR, EXPR] .agree_nok_TCCl 
consensus_tcc [EXPR, EXPR] . agree_ok_TCCl 
consensus_tcc [EXPR, EXPR] . Clprop_r_TCCl 
consensus_tcc [EXPR, EXPR] . Cl_f inal_proof _TCC1 


Use of the formula 
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induct ion . limited.induct ion 
requires the following TCCs to be proven 
induct ion_tcc . ind.m.proof.TCCl 

Use of the formula 

noetherian[naturalnuraber , induction. prev] .general_induction 
requires the following assumptions to be discharged 

no ether ian[naturalnumber, induct ion. prev] . well_f ounded 

================== SUMMARY ================== 

The proof chain is complete 

The axioms and assumptions at the base are: 
consensus [EXPR, EXPR] . car d_ remove. ax 
consensus [EXPR, EXPR] . fullset_card_ax 
consensus [EXPR, EXPR] ,maj_ext 
consensus [EXPR, EXPR].majax 
consensus [EXPR, EXPR] .mn.prop 
consensus [EXPR, EXPR] .non.empty_ax 
consensus [EXPR, EXPR] . send_ax 
functionprops [EXPR, EXPR] . extensionality 
noetherian[EXPR, EXPR] .general_induction 
Total: 9 

The definitions and type-constraints are: 
consensus [EXPR, EXPR].Clprop 
consensus [EXPR, EXPR] . C2prop 
consensus [EXPR, EXPR].0M 
consensus [EXPR, EXPR] . faulty .members 
naturalnumbers ,nat_ invariant 
Total : 5 


The formulae used 
consensus [EXPR, 
consensus [EXPR, 
consensus [EXPR, 
consensus [EXPR, 
consensus [EXPR, 
consensus [EXPR, 
consensus [EXPR , 
consensus [EXPR, 
consensus [EXPR, 
cons ensus [EXPR , 
consensus [EXPR, 
consensus [EXPR, 


are : 

EXPR] .Cl 
EXPR] . Cl_f inal 
EXPR] . Clprop.O 
EXPR] . Clprop.r 
EXPR] . C2 
EXPR] . C2prop_0 
EXPR] . C2prop_r 
EXPR] . OMO.ok 
EXPR] . OMO.prop 
EXPR] . OM.prop 
EXPR] . agree.nok 
EXPR] . agree.ok 
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Appendix C. Tie Full Specification and Verification 


consensus [EXPR, EXPR].all_ok 
consensus [EXPR, EXPR] . distr_prop 

consensus [EXPR , EXPR] . f aulty_members_carcL_remove_nok 

consensus [EXPR , EXPR] . fault y -members. card_remove_ok 

consensus [EXPR , EXPR] . ok.card.remove 

consensus [EXPR, EXPR] . ok_others 

consensus [EXPR, EXPR] .ok.self 

consensus [EXPR, EXPR] .remove.nok 

consensus [EXPR, EXPR] .remove.nok.member 

consensus [EXPR, EXPR] .remove.ok 

consensus [EXPR, EXPR] .remove-ok.member 

consensus [EXPR, EXPR] . remove_others 

consensus [EXPR, EXPR] . round. induct 

cons ensus.tcc [EXPR, EXPR] . Cl_f inal_TCCl 

cons ensus_tcc [EXPR, EXPR] . Cl.f inal .proof _TCC1 

consensus_tcc [EXPR, EXPR] . Clprop.r.TCCl 

consensus.tcc [EXPR, EXPR] . C2prop.r_TCCi 

consensus.tcc [EXPR, EXPR] . OHO.ok.TCCl 

consensus.tcc [EXPR, EXPR] . OMO.prop.TCCl 

consensus__tcc [EXPR, EXPR] .0M_TCC1 

consensus.tcc [EXPR, EXPR] .0M_TCC2 

consensus.tcc [EXPR, EXPR] . OM.prop.TCCl 

consensus.tcc [EXPR, EXPR] . agree.nok.TCCl 

consensus.tcc [EXPR , EXPR] . agree.ok.TCCl 

consensus_tcc [EXPR, EXPR] . next.round.TCCl 

cons ensus.tcc [EXPR, EXPR] . ok.others.TCCl 

consensus.tcc [EXPR , EXPR] .processors_TCCl 

consensus.tcc [EXPR, EXPR] .round, induct _TCCi 

consensus.tcc [EXPR, EXPR] .round. induct_TCC2 

consensus_tcc [EXPR, EXPR] .round. induct. pr oof _TCC1 

consensus_tcc[EXPR, EXPR] . round. induct.proof.TCC2 

consensus_tcc [EXPR, EXPR] .rounds.TCCl 

induction .basic.induct ion 

induction. induct ion_m 

induction. limited-induction 

induction_tcc . ind.m.proof.TCCl 

noetherian[naturalnumber , induction. prev] .well-founded 

Total: 49 

The completed proofs are: 

consensus [EXPR, EXPR] . Cl.f inal.proof 
consensus [EXPR, EXPR] . Cl.proof 
consensus [EXPR, EXPR] . Clprop_0_proof 
consensus [EXPR, EXPR] .Ciprop_r_proof 
consensus [EXPR, EXPR] . C2_proof 
consensus [EXPR, EXPR] . C2prop.0_proof 


C.2. Proof-Chain Analysis 
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consensus [EXPR , EXPR] . C2prop_r.proof 
consensus [EXPR, EXPR] . OMO.ok.proof 
consensus [EXPR, EXPR] . OMO.prop.proof 
consensus [EXPR, EXPR] . OM.prop.proof 
consensus [EXPR, EXPR] . agree.nok.proof 
consensus [EXPR, EXPR] . agree.ok.proof 
consensus [EXPR, EXPR] . all.ok.proof 
consensus [EXPR, EXPR] . distr.prop.proof 

consensus [EXPR, EXPR] . fault y .members .card.remove.nok.proof 
consensus [EXPR, EXPR] . fault y .members. card.remove.ok.proof 
consensus [EXPR, EXPR] . ok.card.remove.proof 
consensus [EXPR, EXPR] . ok.others.proof 
consensus [EXPR, EXPR] . ok_self .proof 
consensus [EXPR , EXPR] . remove.nok.raember.proof 
consensus [EXPR, EXPR] . remove.nok.proof 
consensus [EXPR, EXPR] . remove _ok_member_pr oof 
consensus [EXPR, EXPR] . remove.ok.proof 
consensus [EXPR, EXPR] . remove.others.proof 
consensus [EXPR, EXPR] .round.induct.proof 
consensus.tcc [EXPR , EXPR] . Cl.f inal.TCCl.PRQOF 
consensus.tcc [EXPR, EXPR] . Cl_f inal.proof.TCCl.PROOF 
consensus.tcc [EXPR, EXPR] . Clprop.r.TCCl.PROOF 
consensus.tcc [EXPR, EXPR] . C2prop.r_TCCl_PR00F 
consensus.tcc [EXPR , EXPR] . OMO.ok.TCCl.PROOF 
consensus.tcc [EXPR, EXPR] . OMO.prop.TCCi.PROOF 
consensus.tcc [EXPR, EXPR] .0M.TCC1. PROOF 
consensus.tcc [EXPR, EXPR] .OM.TCC2.PROOF 
consensus.tcc [EXPR, EXPR] . OM.prop.TCCl .PROOF 
consensus.tcc [EXPR, EXPR] . agree.nok.TCCi.PROOF 
consensus_tcc[EXPR, EXPR] . agree.ok.TCCl. PROOF 
consensus_tcc [EXPR, EXPR] .next.round.TCCl.PROOF 
consensus.tcc [EXPR, EXPR] . ok.others.TCCi.PROOF 
consensus.tcc [EXPR , EXPR] .round.induct.TCCl.PROOF 
consensus_tcc [EXPR , EXPR] .round. induct. TCC2.PR00F 
consensus.tcc [EXPR , EXPR] . round.induct.proof .TCC1.PR00F 
consensus.tcc [EXPR, EXPR] .round. induct .proof .TCC2.PR00F 
induct ion . discharge 
induction. ind.m.proof 
induction. ind.proof 
induction . limited.proof 
induct ion.tcc . ind.m.proof .TCC1.PR00F 
top [EXPR, EXPR] .processors.TCCl.PROOF 
top [EXPR, EXPR] .rounds.TCCl.PROOF 
Total: 49 
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